{
    "cells": [
        {
            "cell_type": "markdown",
            "id": "9693dcaf-edc6-46e6-9371-f2571a4a1b3c",
            "metadata": {},
            "source": [
                "# Deplot Reader Demo\n",
                "\n",
                "In this notebook we showcase the capabilities of our ImageTabularChartReader, which is powered by the DePlot model https://arxiv.org/abs/2212.10505."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "id": "9714bf99-3ca0-4353-8e0c-20afd3470ac3",
            "metadata": {},
            "outputs": [],
            "source": [
                "!pip install llama-hub"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 13,
            "id": "17bc9aca-456b-4e0a-95cc-bcaa70ffb833",
            "metadata": {
                "tags": []
            },
            "outputs": [],
            "source": [
                "from llama_hub.file.image_deplot.base import ImageTabularChartReader\n",
                "from llama_index import SummaryIndex\n",
                "from llama_index.response.notebook_utils import display_response\n",
                "from pathlib import Path"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 14,
            "id": "08c00e87-81d0-462e-9b35-4149c9492983",
            "metadata": {
                "tags": []
            },
            "outputs": [],
            "source": [
                "loader = ImageTabularChartReader(keep_image=True)"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "2a329200-bf9b-43d4-a5a2-8255cd3b4db3",
            "metadata": {},
            "source": [
                "## Load Protected Waters Chart\n",
                "\n",
                "This chart shows the percentage of marine territorial waters that are protected for each country."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 15,
            "id": "eb74c5f3-439e-440e-8955-eac5dbdb2a9d",
            "metadata": {
                "tags": []
            },
            "outputs": [],
            "source": [
                "documents = loader.load_data(file=Path(\"./marine_chart.png\"))"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 16,
            "id": "73b26b5a-5d95-4e8d-8ebd-565085df7be2",
            "metadata": {
                "tags": []
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Figure or chart with tabular data: Country | Share of marine territorial waters that are protected, 2016 <0x0A> Greenland | 4.52 <0x0A> Mauritania | 4.15 <0x0A> Indonesia | 2.88 <0x0A> Ireland | 2.33\n"
                    ]
                }
            ],
            "source": [
                "print(documents[0].text)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 17,
            "id": "225a1330-addf-4465-b368-23e62afe4d65",
            "metadata": {
                "tags": []
            },
            "outputs": [
                {
                    "name": "stderr",
                    "output_type": "stream",
                    "text": [
                        "Retrying langchain.llms.openai.completion_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised APIConnectionError: Error communicating with OpenAI: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')).\n"
                    ]
                }
            ],
            "source": [
                "summary_index = SummaryIndex.from_documents(documents)\n",
                "response = summary_index.as_query_engine().query(\n",
                "    \"What is the difference between the shares of Greenland and the share of Mauritania?\"\n",
                ")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "id": "ed97789d-8907-40ed-86f8-309c2634a090",
            "metadata": {
                "tags": []
            },
            "outputs": [],
            "source": [
                "display_response(response, show_source=True)"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "81a66b05-f1f9-4a6f-80ce-b661688a6f21",
            "metadata": {},
            "source": [
                "## Load Pew Research Chart\n",
                "\n",
                "Here we load in a Pew Research chart showing international views of the US/Biden.\n",
                "\n",
                "Source: https://www.pewresearch.org/global/2023/06/27/international-views-of-biden-and-u-s-largely-positive/"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 19,
            "id": "64835171-5e0d-4e86-a7d3-63e5710a18d3",
            "metadata": {
                "tags": []
            },
            "outputs": [],
            "source": [
                "documents = loader.load_data(file=Path(\"./pew1.png\"))"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 20,
            "id": "9cc2b5b3-e413-45a9-8980-699701371251",
            "metadata": {
                "tags": []
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Figure or chart with tabular data: Entity | Values <0x0A> Does not | 50.0 <0x0A> % who say the U.S take into account the interests of countries like theirs | 49.0 <0x0A> Does not | 38.0 <0x0A> % who say the U.S contribute to peace and stability around the world | 61.0 <0x0A> Does not | 15.0 <0x0A> % who say the U.S interfere in the affairs of other countries | 15.0 <0x0A>% who have confidence | 54.0 <0x0A> Views of President Biden | 30.0 <0x0A> Favorable | 59.0 <0x0A> Views of the U.S. | 9.0\n"
                    ]
                }
            ],
            "source": [
                "print(documents[0].text)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 21,
            "id": "71574aae-df92-49a5-b251-e3bab4b8de3f",
            "metadata": {
                "tags": []
            },
            "outputs": [],
            "source": [
                "summary_index = SummaryIndex.from_documents(documents)\n",
                "response = summary_index.as_query_engine().query(\n",
                "    \"What percentage says that the US contributes to peace and stability?\"\n",
                ")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "id": "2a5a5afd-af6c-4781-9ea5-82d206797cbb",
            "metadata": {
                "tags": []
            },
            "outputs": [],
            "source": [
                "display_response(response, show_source=True)"
            ]
        }
    ],
    "metadata": {
        "kernelspec": {
            "display_name": "llama_index_v2",
            "language": "python",
            "name": "llama_index_v2"
        },
        "language_info": {
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "file_extension": ".py",
            "mimetype": "text/x-python",
            "name": "python",
            "nbconvert_exporter": "python",
            "pygments_lexer": "ipython3",
            "version": "3.10.10"
        }
    },
    "nbformat": 4,
    "nbformat_minor": 5
}
